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Abstract 

Background: Various studies have assessed the diagnostic accuracy of EGFR mutation-specific antibodies in non-small cell 
lung cancer (NSCLC). We performed a meta-analysis of existing data to investigate the diagnostic value of mutation-specific 
antibodies for detection of EGFR mutations in NSCLC. 

Methods: We systematically retrieved relevant studies from PubMed, Web of Knowledge, and Google Scholar. Data from 
studies that met the inclusion criteria were extracted for further exploration of heterogeneity, including calculation of the 
average sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR), 
and analysis of SROC(summary receiver operating characteristic) curves. 

Results: Fifteen studies met our inclusion criteria. A summary of the meta-analysis of the efficacy of the anti-E746-A750 
antibody was as follows: sensitivity, 0.60 (95% CI, 0.55-0.64); specificity, 0.98 (95% CI, 0.97-0.98); PLR, 33.50 (95% CI, 13.96- 
80.39); NLR, 0.39 (95% CI, 0.30-0.51) and DOR, 1 1 1.17 (95% CI, 62.22-198.63). A similar meta-analysis was performed for the 
anti-L858R antibody with results as follows: sensitivity, 0.76 (95% CI, 0.71-0.79); specificity, 0.96 (95% CI, 0.95-0.97); PLR, 
24.42 (95% CI, 11.66-51.17); NLR, 0.22 (95% CI, 0.12-0.39) and DOR, 126.66 (95% CI, 54.60-293.82). 

Conclusion: Immunohistochemistry alone is sufficient for the detection of EGFR mutations if the result is positive. 
Molecular-based analyses are necessary only if the anti-E746-A750 antibody results are negative. Immunohistochemistry 
seems more suitable for clinical screening for EGFR mutations prior to molecular-based analysis. 
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Introduction 

Lung cancer is the most frequent cause of cancer-related death 
worldwide [1]. Non-small cell lung cancer (NSCLC) makes up 
approximately 80% of lung cancer and is rapidly becoming one of 
the major diseases that threatens human health. Somatic 
mutations in the epidermal growth factor receptor (EGFR) gene 
are found in approximately 10%— 16% of NSCLC patients in 
United States and Europe [2] and 30%-50% of patients in Asia 
[3]. The two most common genetic mutations are the in-frame 
deletion in exon 19 (E746-A750) and the substitution of leucine 
858 by arginine in the exon 21(L858R) [4]. These two mutations 
constitute about 90% of all mutations and are known as the 
"classical" mutations [5] . These two EGFR-specific mutations are 
strong predictors of the response to small-molecule EGFR- tyrosine 
kinase inhibitors such as gefitinib [6,7] and erlotinib [8]. 

Direct DNA sequencing is a classical method for EGFR 
mutation detection. However, expensive equipment and amount 



of time are necessary for this technique. Furthermore, it is difficult 
to extract the required amounts of high quality DNA from pure 
tumor cells, which limits direct sequencing in clinical usage. 
Recently, several other molecular-based analyses have been 
developed to detect EGFR mutations, including the Scorpion 
amplification refractory mutation system (ARMS), Smart Ampli- 
fication Process (SMAP), polymerase chain reaction-single strand 
conformation polymorphism (PCR-SSCP), and high resolution 
melting analysis (HRMA), etc. These novel methods require less 
tumor tissue and less time while achieving high sensitivities and 
specificities. However, they require advanced operating skills and 
sophisticated equipment, which hampers their application in 
clinical practice. 

Therefore, it would be beneficial to find an easy, cost-effective, 
and accurate method to identify EGFR-mutations in NSCLC. Use 
of immunohistochemistry (IHC) to identify mutant EGFR proteins 
via specific antibodies is an example of such a method. Yu et al [9] 
immunized New Zealand rabbits with synthetic peptides matching 
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the EGFR sequence with the E746-A750 deletion in exon 19 or 
the L858R point mutation in exon 21. By contrast, conflicting 
results are reported by several recent studies on the potential 
diagnostic value of mutation-specific antibodies for immunohisto- 
chemical detection of EGFR mutations in NSCLC. For instance, 
the sensitivity of anti-E746-A750 antibody was 36% reported by 
Hofman et al [10] while it reached 100% in Hasanovic et al study 
[11]. 

In order to clarify the value of mutation-specific antibodies in 
the identification of EGFR mutation status, a meta-analysis was 
conducted to systematically and quantitatively evaluate the 
accuracy of the immunohistochemical method for EGFR muta- 
tion screening in NSCLC. 

Material and Methods 

Data sources and searches 

We identified relevant studies by searching PubMed, Web of 
Knowledge, and Google Scholar. We limited our search to English 
language literature published between May 2009 and July 2013. 
The keywords used included 'immunohistochemistry', 'EGFR 



mutation', 'NSCLC, 'non-small cell lung cancer', 'lung carcino- 
ma', 'lung adenocarcinoma', 'pulmonary adenocarcinoma', and 
'mutation-specific antibodies'. Articles were also identified by use 
of the related articles function in PubMed. 

Two reviewers (Zi Chen and Hong-bing Liu) inspected the title 
and abstract of each citation independently to identify those 
studies that were likely to report the diagnostic value of EGFR 
mutation-specific antibodies. For those articles that were not 
excluded based on title and abstract, reviewers retrieved full text, 
made judgment and decided final conclusion for them. If 
disagreement occurred, two reviewers discussed and arrived at 
consensus (Zi Chen and Hong-bing Liu). Inclusion criteria for the 
primary studies were as follows: (1) all samples were NSCLC, 
confirmed either histologically or cytologically; (2) must have used 
the authoritative molecule-based standard for the EGFR mutation 
and immunohistochemical staining score criteria. (3) results in 
each individual study could be summarized in a 2 X 2 contingency 
table; and (4) there were no restrictions as to data collection timing 
(i.e., prospective or retrospective). 



Records identified through 
database searching 
(n = 282) 



Records screened 
(n = 222) 



Full-text articles assessed 
for eligibility 
(n = 25) 



Studies included in 
quantitative synthesis 
(meta-ana lysi s) 
(n = 15) 



Duplicates removed 
(n = 60) 



Records excluded by title and abstract 
(n = 197) 



10 excluded for the following reason: 
Did not report immunohistochemical 
staining score criteria =2 
Could not retrieve 2x2 contingency 
table = 3 

Did not study in NSCLC patients = 2 
Non-English literature = 3 



Figure 1. Flow diagram of study selection by using electronic database. 

doi:10.1371/journal.pone.0105940.g001 
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Table 3. Characteristics of included studies. 





Reference/year 


Blinded design 


Consecutive or random 


Tissue sample preparation 


Quality score 












STARD 


QUADAS 


Yu et al (2009) [9] 


Yes 


Unknown 


FFPE 


10 


11 


Brevet et al (2010) [24] 


Unknown 


Yes 


FFPE 


13 


11 


Simonetti et al (2010) [31] 


No 


Unknown 


FFPE 


10 


12 


Kawahara et al (2010) [28] 


Unknown 


Yes 


FFPE 


12 


11 


Kitamura et al (2010) [29] 


No 


Yes 


FFPE/frozen 


12 


11 


Hie et al (2010) [26] 


Unknown 


Yes 


FFPE/frozen 


14 


10 


Kozu et al (2010) [30] 


Unknown 


Yes 


MFPE/frozen 


15 


12 


Kato et al (2010) [27] 


Unknown 


Unknown 


FFPE 


15 


12 


Hofman et al (2012) [10] 


Yes 


Yes 


FFPE 


12 


10 


Wu et al (2011) [32] 


Unknown 


Yes 


FFPE 


19 


12 


Hasanovic(2012) [11] 


Unknown 


Yes 


FFPE 


13 


8 


Ambrosini-Spaltro et al (2012) [22] 


Unknown 


Yes 


FFPE 


9 


12 


Angulo et al (2012) [23] 


Unknown 


Yes 


FFPE 


12 


10 


Cooper et al (2013) [25] 


Yes 


Yes 


FFPE 


17 


13 


Xiong et al (2013) [33] 


Unknown 


Yes 


FFPE 


17 


11 



Abbreviations: FFPE, formalin-fixed paraffin-embedded; MFPE, methanol-fixed paraffin-embedded; STARD, the Standards for Reporting Diagnostic accuracy, QUADAS, 
the Quality Assessment of Diagnostic Accuracy Studies. 
doi:1 0.1 371 /joumal.pone.01 05940.t003 



Review articles, editorials, case reports, and corresponding 
letters were excluded due to lack of original data. If we found 
multiple articles for a single study, we used the best-quality one. 

Data extraction and quality assessment 

Two investigators (Zi Chen and Hong-bing Liu) extracted the 
following data from independently the selected studies: (1) year of 
publication; (2) location of the study; (3) number of tumor tissue or 
cytology specimens; (4) IHC methodology; (5) IHC score criteria; 
(6) standard; (7) number of true positive (TP); (8) number of false 
positive (FP); (9) number of false negative (FN), and (10) number of 
true negative (TN) for the exon 19 deletion and exon 21 L858R 
mutation, respectively. In addition, for an accurate evaluation of 
heterogeneity, the following characteristics of study design were 
retrieved: (1) whether the study was double-blind regarding the 
results of the immunohistochemical method and the results of the 
molecule-based analysis; (2) whether there was consecutive or 
random sampling of patients; and (3) tissue sample preparation 
[whether FFPE (Formalin-Fixed Paraffin-Embedded) was used]. 
The Quality Assessment of Diagnostic Accuracy Studies (QUA- 
DAS, maximum score 14) [12] and the Standards for Reporting 
Diagnostic accuracy (STARD, maximum score 25) [13] were used 
to assess the quality of the selected studies. Disagreements were 
resolved by discussion between Zi Chen and Hong-bing Liu. 

Statistical analysis 

We used standard methods recommended for meta-analysis of 
diagnostic test evaluations [14]. Firstiy, we tested for the presence 
of cut-off point effects. Estimates of diagnostic accuracy differ if 
not all studies use the same cut-off point for a positive test result or 
for the reference standard. Variation in the parameters of 
accuracy may be partly due to variation in cut-off point. We 
tested for the presence of a cut-off point effect between studies by 
calculating a Spearman correlation coefficient between sensitivity 
and specificity of all included studies [14]. A positive rank- 



correlation coefficient and a p<0.05 are suggestive of a significant 
cut-off point effect. If the cut-off point effect was present, the 
sensitivity, specificity, LR and DOR of each research were not 
suitable for merger. 

A SROC curve was the basis of the meta-analysis [14,15]. The 
SROC curve was plotted to identify the sensitivity and specificity 
for the single test threshold from each study [15,16]. We calculated 
the respective area under SROC curve and Q* index on SROC 
curve where sensitivity equals to specificity. A random-effects 
model (REM) was used to calculate the average sensitivity, 
specificity, positive likelihood ratio (PLR), negative likelihood ratio 
(NLR) and diagnostic odds ratio (DOR) [17]. 

The DOR is a common comprehensive evaluation indicator, 
which combines data from sensitivity, specificity, PLR and NLR 
into a single number: (TP/FN)/(FP/TN) [18]. The DOR of a test 
is the ratio of the odds of positive test results in NSCLC patients 
with EGFR mutations relative to the odds of positive test results in 
the wild-type patients. The value of a DOR ranges from 0 to 
infinity, with higher values implying better discriminative test 
performance. 

In this meta-analysis, besides cut-off point effect, there were 
additional factors that can cause heterogeneity as well. Majority 
diagnostic reviews indicate considerable heterogeneity in the 
results of included studies [14]. When different studies have largely 
different results, this may result from either random error or 
heterogeneity due to differences in clinical or methodological 
characteristics of studies [14]. We used the I 2 test for the pooled 
DOR (PDOR) to detect statistically significant heterogeneity [19]. 
The PDOR was computed according to standard methods to 
analyze the changing in diagnostic accuracy in the study per unit 
in the covariates [20]. I &50% indicated substantial heterogene- 
ity. We included the STARD and the QUADAS as covariates in 
univariate meta-regression analysis by assessing the effects of their 
score on the diagnostic ability of mutation-specific antibodies. We 
also analyzed the effects of other covariates on blinded design, 
consecutive or random specimen of patients, IHC methodology, 
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Sensitivity (95% CI) 


Yu 


0.88 


(0.70 


- 0.98) 


Brevet 


0.74 


(0.55 


- 0.88) 


Simonetti 


0.69 


(0.49 


- 0.85) 


Kawahara 


0 62 


(0.38 


- 0.82) 


Kitamura 


0.39 


(0.24 


- 0.55) 


Hie 


0.90 


(0.55 


- 1.00) 


Kozu 


0.42 


(0.34 


-0.51) 


Kato 


0.50 


(0.26 


- 0.74) 


Hofman 


0.36 


(0.21 


- 0.54) 


Wu 


0.73 


(0.57 


- 0.86) 


Hasanovic 


1.00 


(0.91 


- 1.00) 


Ambrosini-Spalto 


0.50 


(0.21 


- 0.79) 


Angulo 


0.73 


(0.39 


- 0.94) 


Cooper 


1.00 


(0.59 


- 1.00) 


Xiong 


0.59 


(0.33 


- 0.82) 



Pooled Sensitivity = 0.60 (0.55 to 0.64) 
Chi-square = 105.84; df = 14 (p = 0.0000) 
Inconsistency (l-square) = 86.8 % 



B 




.2 



.4 .6 
Specificity 





Specificity (95% CI) 


Yu 


1.00 


(0.98- 1.00) 


Brevet 


0.99 


(0.96- 1.00) 


Simonetti 


1.00 


(0.93- 1.00) 


Kawahara 


1.00 


(0.91 - 1.00) 


Kitamura 


0.99 


(0.97- 1.00) 


Hie 


0.92 


(0.81 - 0.98) 


Kozu 


1.00 


(0.98- 1.00) 


Kato 


1.00 


(0.93- 1.00) 


Hofman 


0.97 


(0.93 - 0.99) 


Wu 


0.95 


(0.89 - 0.98) 


Hasanovic 


0.77 


(0.68-0.85) 


Ambrosini-Spalto 


1.00 


(0.84- 1.00) 


Angulo 


1.00 


(0.95- 1.00) 


Cooper 


0.98 


(0.95 - 0.99) 


Xiong 


1.00 


(0.89- 1.00) 



Pooled Specificity = 0.98 (0.97 to 0.98) 
Chi-square = 115.90; df = 14 (p = 0.0000) 
Inconsistency (l-square) = 87.9 % 



Figure 2. Forest plots for sensitivity (A) and specificity (B) of the anti-E746-A750 antibody in the detecting the EGFR exon 19 

deletion. Sensitivity =0.60 (95% CI, 0.55-0.64); specificity =0.98 (95% CI, 0.97-0.98). 

doi:10.1371/journal.pone.0105940.g002 



IHC score criteria, and standard. Subgroup analysis was 
performed to explore the sources of potential heterogeneity 
among studies using univariate meta-regression analysis. As 
publication bias is of concern for the meta-analyses of diagnostic 
studies, we tested for the potential presence of this bias using 
Deeks' funnel plots. [21] 

All analyses were performed using two statistical software 
programs, Stata, version 12.0 (Stata Corporation, College Station, 
TX, USA) and Meta-Disc 1.4 for Windows (XI Cochrane 
Colloquium, Barcelona, Spain). The statistical significance was 
set at p<0.001. 



Results 

Eligible studies and quality assessment 

As shown in figure 1, the literature search identified fifteen 
published studies[9-l 1,22-33] that fulfilled the inclusion criteria. 
Summaries and characteristics of these studies are reported in 
table 1-3. Overall, 2337 cases were entered in the meta-analysis, 
ranging from 33 to 577 patient specimens per study. As shown in 
table 1 and table 2, seven of the fifteen studies (47%) used tissue 
microarray technology in the immunohistochemical method; 
twelve studies (80%) set the four grades of visual scoring as the 
IHC score criteria; in twelve studies (80%) direct sequencing was 
used as the standard. As shown in the table 3, three of fifteen 
studies (20%) used a double-blind study design while evaluating 
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Sensitivity (95% CI) 


Yu 


0.92 


(0.75 


- 0.99) 


Brsvst 


0.95 


(0.76 


- 1.00) 


^imonptti 

Oil 1 l\Ji IKZlli 


0.93 


(0.76 


- 0.99) 


Kawahara 


0.83 


(0.61 


- 0.95) 


Kitami ira 
rxiiai I mi a 


0.32 


(0.18 


- 0.50) 


Kozu 


0.76 


(0.68 


- 0.82) 


Kato 


0.75 


(0.43 


-0.95) 


Hofman 


0.19 


(0.05 


- 0.42) 


Wu 


0.88 


(0.75 


- 0.96) 


O Hasanovic 


1.00 


(0.88 


- 1.00) 


Ambrosini-Spalto 


0.83 


(0.36 


- 1.00) 


Angulo 


0.40 


(0.05 


- 0.85) 


- Cooper 


0.86 


(0.42 


- 1.00) 


Xiong 


0.81 


(0.54 


- 0.96) 



Pooled Sensitivity = 0.76 (0.71 to 0.79) 
Chi-square = 101.80; df = 13 (p = 0.0000) 
1 Inconsistency (l-square) = 87.2 % 



B 



Specificity (95% CI) 



Yu 

Brevet 

Simonetti 

Kawahara 

Kitamura 

Kozu 

Kato 

Hofman 

Wu 

Hasanovic 

Ambrosini-Spalto 

Angulo 

Cooper 

Xiong 



0.99 
0.99 
1.00 
0.97 
0.97 
0.98 
0.97 
0.98 
0.77 
0.87 
1.00 
1.00 
0.98 
0.97 



(0.96 ■ 
(0.96 • 
(0.93 
(0.86 • 
(0.94 
(0.96 
(0.88 ■ 
(0.94 ■ 
(0.68 ■ 
(0.80 ■ 
(0.87 
(0.95 ■ 
(0.96 ■ 
(0.85 • 



1.00) 
1.00) 
1.00) 
1.00) 
0.99) 
0.99) 
1.00) 
1.00) 
0.85) 
0.93) 
1.00) 
1.00) 
1.00) 
1.00) 



.4 .6 
Specificity 



Pooled Specificity = 0.96 (0.95 to 0.97) 
Chi-square = 96.89; df = 13 (p = 0.0000) 
1 Inconsistency (l-square) = 86.6 % 



Figure 3. Forest plots for sensitivity (C) and specificity (D) of the anti-L858R antibody in the detecting the EGFR exon 21 mutation. 

Sensitivity =0.76 (95% CI, 0.71-0.79); specificity =0.96 (95% CI, 0.95-0.97). 
doi:1 0.1 371 /journal.pone.01 05940.g003 



the accuracy of detecting EGFR mutation status by molecular- 
based analyses compared with the immunohistochemical method. 
In twelve studies (80%), the specimens were collected from 
consecutive or randomized patients. The NSCLC was confirmed 
by histological and cytological examination. The characteristics, 
together with the STARD and the QUADAS scores of these 
studies are outlined in table 3. 

Heterogeneity assessment and diagnostic accuracy 

The Spearman correlation coefficient of the anti-E746-A750 
antibody and the anti-L858R antibody were 0.360 (P = 0. 187) and 
— 0.033 (P = 0.911), respectively, verifying that the variability 
across these studies could not be explained by differences in the 
diagnostic cut-off point (since the P-values were not <0.05). 

Figure 2 and 3 shows forest plots of sensitivity and specificity for 
the anti-E746-A750 antibody and the anti-L858R antibody in the 



identification of EGFR mutation status. For theanti-E746-A750 
antibody, the sensitivity ranged from 0.36 to 1.00 (mean, 0.60; 
95% confidence interval (CI), 0.55-0.64), while specificity ranged 
from 0.77 to 1.0 (mean, 0.98; 95% CI, 0.97-0.98). For the anti- 
L858R antibody, the sensitivity ranged from 0.19 to 1.00 (mean, 
0.76; 95% CI), 0.71-0.79), while specificity ranged from 0.77 to 
1.0 (mean, 0.96; 95% CI, 0.95-0.97). In figure 4, we also noted 
that the PLR, NLR, and DOR of the anti-E746-A750 antibody 
were 33.50 (95% CI, 13.96-80.39), 0.39 (95% CI, 0.30-0.51), and 
111.17 (95% CI, 62.22-198.63), respectively; the PLR, NLR, and 
DOR of the anti-L858R antibody were 24.42 (95% CI, 11.66- 
51.17), 0.22 (95% CI, 0.12-0.39), and 126.66 (95% CI, 54.60- 
293.82), respectively (figure 5). For the anti-E746-A750 antibody, 
the r test for PDOR was 12.6%, which did not show any major 
qualitative evidence for heterogeneity between studies. Regarding 
PLR and NLR, we found significant heterogeneity for all of the 
inclusion studies, I 2 = 84.6% and 78.6%, respectively. For the anti- 
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Figure 4. Forest plot for the positive likelihood ratio (PLR) (A), the negative likelihood ratio (NLR) (B) and the diagnostic odds ratio 
(DOR) (C) of the anti-E746-A750 antibody. PLR (positive likelihood ratio) =33.50 (95% CI, 13.96-80.39); NLR (negative likelihood ratio) =0.39 
(95% CI, 0.30-0.51); DOR (diagnostic odds ratio) =111.17 (95% CI, 62.22-198.63). 
doi:10.1371/journal.pone.0105940.g004 



L858R antibody, the I 2 test for PDOR, PLR, and NLR were 
66.4%, 85.7%, and 93.6%, respectively, which showed substantial 
heterogeneity among studies. 

Besides indicating the degree of equivalency between sensitivity 
and specificity, the SROC curve and the area under the curve also 
give a general summary of performance. Graphs of the SROC 
curves for the anti-E746-A750 antibody and the anti-L858R 
antibody the rate of true-positives compared with the rate of false- 
positives from individual studies are shown in figure 6. For the 
anti-E746-A750 antibody, the area under the curve (AUC) was 
0.971 1 (Q* index = 0.9216); for the anti-L858R antibody, the area 
under the curve (AUC) was 0.9800 (Q* index =0.9371). These 
data indicate that both mutation-specific antibodies represent a 
high level overall accuracy. 

Meta regression and sub-group analysis 

A quality score for every study was completed using STARD 
guidelines [1 3] , in which each score is compiled on the basis of title 
and introduction, methods, results and discussion (table 3). The 
QUAD AS tool [12] was also used to scale the score by 1, when a 
criterion was fulfilled; 0, if a criterion was unclear and — 1 , if the 
criterion was not achieved (table 3). These scores were employed 
in the meta-regression analysis to evaluate the effect of study 
quality on the PDOR of mutation-specific antibodies in identifi- 
cation of the EGFR mutation status. 

For the anti-E746-A750 antibody, as shown in table 4, studies 
with lower quality (STARD score<13; QUAD AS score<10) had 
PDOR values that were not obviously lower than those of studies 
of higher quality. We also noted that differences among studies 
with or without blinded design, consecutive or random design, 
IHC methodology [tissue microarray (TMA) vs. individual slides], 
IHC score criteria (consider 2+ or 3+ as positive vs. others), and 
standard used for immunohistochemical method (direct sequenc- 
ing vs. others) did not reach statistical significance, which implies 
that the diagnostic accuracy was not substantially affected by the 
design of study. 

For the anti-L858R antibody, as shown in table 4, we noticed 
that differences quality, blinded design, and IHC methodology 
among studies did not contribute to the heterogeneity. However, 
the differences among consecutive or random design, IHC score 
criteria, and standard reached statistical significance, which 
indicates that the study design could affect the diagnostic accuracy. 

According to the results of meta-regression, we performed a 
sub-group analysis, and the data was show in table 5 and table 6. 

Evaluation of publication bias 

The Deek's funnel plot asymmetry test revealed the lack of 
publication bias (the anti-E746-A750 antibody, P = 0.93; the anti- 
L858R antibody, P = 0.85) (figure 7) [21]. 

Discussion 

In recent years, two novel antibodies for IHC have been 
developed against the most common EGFR mutations, the 15 bp 
exon 19 deletions and the L858R mutation in exon 21 [9]. The 
discovery of mutation-specific antibodies opened up a new 
possibility for the detection of the EGFR mutation in NSCLC. 
Many studies have been done in order to evaluate its diagnostic 
power of these antibodies; however, there has been no consensus 



as to their efficacy. Therefore, in the present study, we analyzed 
the data by meta-analysis to obtain an accurate conclusion. 

The present meta-analysis demonstrates that the L858R 
antibody has higher sensitivity than the E746-A750 antibody 
(76% vs. 60%). Considering the sensitivity of the anti-E746-A750 
antibody is only 60%, it will increase the risk of a false negative if 
the pathology technician uses immunohistochemical methods as 
the sole means of detecting the EGFR exon 19 deletion. As the 
anti-E746-A750 antibody specifically detects 15-bp deletions, it 
naturally shows extremely high sensitivity and specificity in 15-bp 
deletion cases. However, the 15-bp exon 19 deletion mutants 
account for only 68.1% of the exon 19 deletions in the COSMIC 
database. Apart from the 15 bp deletions, other exon 19 deletions 
of sizes 9, 12, 18, or 24-bp occur in NSCLC resulting in slighdy 
different epitopes with deletions of 3-8 amino acids. For non- 15- 
bp exon 19 deletion mutants, the sensitivity varied depending on 
the deletion size, ranging from 20% to 67% [24]. Originally, Yu et 
al. reported IHC results on only two non- 15-bp deletion cases, of 
which one was positive by IHC [9] . In Kato et al, all of the exon 
19 deletion samples contained seven non- 15-bp deletion cases, 
none of which were positive using the antibody [27]. However, in 
the chosen 15 studies, the L858R mutation was found in the vast 
majority of exon 21 mutations, which resulted in a moderately 
high sensitivity. However, based on our high specificity (the anti- 
E746-A750 antibody: 99% vs. the anti-L858R antibody: 98%), a 
positive result could eliminate the need for confirmatory molecular 
testing. 

From the Fig2A and 3A, we also found there is a large variation 
in sensitivity for the two antibodies among the 15 chosen studies. 
We considered this was due to a limitation of IHC to EGFR 
mutation testing that only uses mutation-specific antibodies for the 
commoner EGFR mutations. Therefore, rarer sensitizing muta- 
tions in EGFR couldn't be identified. The two most frequent 
mutations in EGFR in NSCLC are the L858R point mutation in 
exon 21, And the proportion of these two types of mutation ranged 
from 52% [24] to 96% [9] of all identified mutations in exon 19 
and 21 among 15 chosen studies Thus, the higher proportion of 
common mutations, the higher sensitivity the mutation specific 
antibodies would be. Kato et al found the overall sensitivity of 
mutation-specific antibodies for detecting EGFR mutations to be 
fairly low (43.9%) when all EGFR mutations were taken into 
account [26] . This result implies the two antibodies are inadequate 
at detecting variant exon 19 deletions and exon 21 point 
mutations. Further refinement of these mutation-specific antibod- 
ies will be required to cover these rare mutations and to improve 
the affinity of these antibodies to the antigen. An antibody cocktail 
could also be developed to detect the common, the rare exon 19 
deletions, exon 21 mutations as well as the resistance mutation 
T790M in exon 20. 

The SROC curve and its AUC do not depend on the diagnostic 
threshold and. In a high quality diagnostic study, the AUC value is 
close to 1 ; however, in low quality studies, the AUC value is close 
to 0.5. The AUC displays a general summary of best performance 
and reveals the equivalency between sensitivity and specificity. In 
our meta-analysis, the maximum joint sensitivity and specificity 
(Q* index) for the anti-E746-A750 antibody is 0.9216 while the 
AUC is 0.9711; for the anti-L858R antibody, the Q* index is 
0.9371 while the AUC is 0.9800. Therefore, the diagnostic 
accuracy of quantitative analysis of mutation-specific antibodies 
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Random Effects Model 

Pooled Diagnostic Odds Ratio = 126.66 (54.60 to 293.82) 
Cochran-Q = 38.70; df = 13 (p = 0.0002) 
Inconsistency (l-square) = 66.4 % 
Tau-squared = 1.4470 



Diagnostic Odds Ratio 

Figure 5. Forest plot for the positive likelihood ratio (PLR) (A), the negative likelihood ratio (NLR) (B) and the diagnostic odds ratio 
(DOR) (C) of the anti-L858R antibody. PLR (positive likelihood ratio) =24.42 (95% CI, 1 1.66-51.17); NLR (negative likelihood ratio) =0.22 (95% CI, 
0.12-0.39); DOR (diagnostic odds ratio) =126.66 (95% CI, 54.60-293.82). 
doi:1 0.1 371 /journal.pone.01 05940.g005 
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with immunohistochemical detection of EGFR mutations is 
similarly effective to molecular-based analyses. 

Compared to the SROC curve, which is not easy to interpret 
and use [34], likelihood ratios are considered to be a more 
meaningful method in clinical practice [35]; therefore, we also 
calculated both PLR and NLR as our detections of diagnostic 
value. In our meta-analysis, the PLR refers to the ratio of the 
probability of mutation-positive results in EGFR mutant-type 
patients (true positive rate, TPR) to the probability of mutation- 
positive results in EGFR wild-type patients (false positive rate, 
FPR). The PLR indicates the probability of positive test result as 
compared to EGFR wild-type patients with the immunohisto- 
chemical method; a larger ratio indicates a higher diagnostic value 
of result. The NLR represents the ratio of the probability of 
mutation-negative results in EGFR mutant-type patients (false 
negative rate, FNR) to the probability of mutation-negative results 
in EGFR wild-type patients (true negative rate, TNR). Unlike the 
PLR, the NLR indicates the probability of a negative test result as 
compared with EGFR wild-type patients with the immunohisto- 
chemical method; therefore, a smaller ratio represents a higher 
diagnostic value of the result. For the anti-E746-A750 antibody, a 
PLR value of 33.50 suggests that NSCLC patients with EGFR 
mutations have about 34-fold higher chance of being IHC-positive 
compared with wild-type patients. This probability strongly 
confirms the diagnosis of EGFR mutation status. For the anti- 
L858R antibody, the PLR value is 24.42; this probability is also 
high enough to confirm the diagnosis of EGFR mutation status. By 
contrast, the NLR of the anti-E746-A750 antibody and the anti- 
L858R antibody were found as 0.39 and 0.22 in the present meta- 
analysis, respectively. If the immunohistochemical result was 
negative, the probability that wild-type patients have mutation 
status is 39% and 22%, respectively. These data illustrate that a 
negative immunohistochemical result should not be used alone as 
a justification to deny mutation status in exon 19; however, for the 
anti-L858R antibody, the result was barely satisfactory. These data 



indicate that for mutations in both in exon 1 9 and exon 2 1 , if the 
immunohistochemical results are positive, the molecular-based 
analysis is not necessary. However, based on intrinsic limitation of 
the sensitivity for the anti-E746-A750 antibody, a negative result 
with this IHC assay could not be used to exclude patients from 
molecular testing. For detection of exon 21 mutation status, 
molecular-based techniques are recommended to reduce the false 
negative rate. 

In our meta-analysis, we have found that the mean DORs were 
111.17 and 126.66 for the anti-E746-A750 antibody and the anti- 
L858R antibody, respectively, which have a high level of overall 
accuracy also. 

Meta-analysis is a comprehensive method for analyzing multiple 
medical studies of the same type and purpose. Pooled data is a 
good option to use when overall included studies are homoge- 
neous. Therefore, an exploration of the reasons for heterogeneity 
is an important goal of meta-analysis [19]. In present meta- 
analysis, an important method to detect heterogeneity was 
evaluated by the I 2 test for the PDOR. Although we found 
statistically significant heterogeneity for sensitivity, specificity, 
PLR, and NLR for the anti-E746-A750 antibody, however, there 
were no heterogeneity between DORs, heterogeneity chi-squared 
= 16.02 (p = 0.3124) and I 2 = 12.6%. For anti-L858R antibody, we 
noticed statistically substantial heterogeneity for DOR, heteroge- 
neity chi-squared = 38.70 (p = 0.0002) and I 2 = 66.4%. In order to 
explore the potential source of heterogeneity, we performed meta- 
regression and sub-group analysis. We did not observe heteroge- 
neity between the higher quality (STARD score >13; QUAD AS 
score &10) and the lower quality studies. Differences among 
studies with or without blinded design and IHC methodology 
(TMA vs. individual slides) did not reach statistical significance. 
However, we noted the differences among studies about consec- 
utive or random design (p = 0.0333), IHC score criteria (consider 
2+ or 3+ as positive vs. others) (p = 0.0262), and standard (direct 
sequencing vs. others) (p = 0.0102) had statistical significance. This 



Table 4. Weighted meta-regression analysis of the effects of methodological and study design on diagnostic value of mutation- 
specific antibodies. 





Covariates 


No. 


Coefficient 


PDOR(95% CI) 


p value 


Anti-E746-A750 antibody 


15 








STARD score >13 


8 


-1.168 


0.31 (0.02;4.88) 


0.3392 


QUADAS score >10 


13 


0.159 


1.17 (0.00; 1 108.68) 


0.9565 


Blinded 


3 


0.610 


1.84 (0.22;15.58) 


0.5105 


Consecutive or random 


12 


- 1 .204 


0.30 (0.03,3.43) 


0.2721 


IHC methodology 


6 


2.488 


12.04 (0.87; 1 66.99) 


0.0598 


IHC score criteria 


8 


2.209 


9.11 (0.88;93.87) 


0.0597 


Standard 


12 


-1.437 


0.24 (0.02,2.89) 


0.2087 


Anti-L858R antibody 


14 








STARD score a 13 


7 


-1.137 


0.32 (0.03;3.11) 


0.2546 


QUADAS score >10 


12 


2.879 


17.81 (0.11, -2840.99) 


0.2043 


Blinded 


3 


1.223 


3.40 (0.52;22.38) 


0.1564 


Consecutive or random 


11 


-2.161 


0.12 (0.02,0.78) 


0.0333 


IHC methodology 


6 


0.680 


1.97 (0.20;19.54) 


0.4805 


IHC score criteria 


7 


2.704 


14.94 (1.61,-138.51) 


0.0262 


Standard 


11 


-2.992 


0.05 (0.01, -0.34) 


0.0102 



Abbreviations: PDOR, pooled diagnostic odds ratio; CI, confidence interval; STARD, the Standards for Reporting Diagnostic accuracy; QUADAS, the Quality Assessment of 
Diagnostic Accuracy Studies; IHC, immunohistochemistry. 
doi:1 0.1 371 /journal.pone.01 05940.t004 
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Figure 7. The funnel plot of publication bias for the anti-E746-A750 antibody (A) and the anti-L858R antibody (B). There was no 
significant publication bias (the anti-E746-A750 antibody, P = 0.93; the anti-L858R antibody, P = 0.85). 
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finding implies that consecutive or random design, IHC score 
criteria, and standard had substantially impact on the diagnostic 
accuracy. In sub-group analysis, based on these three sources of 
heterogeneity, we set up six subgroups for E740-A750 and L858R 
respectively to further explore the heterogeneity. For studies in 2+ 
or 3+ staining as positive sub-group for L858R, we observed the 
numbers of sensitivity, PLR, NLR, and DOR of sub-group 
significandy surpassed the others sub-group with significandy 
decreased consistency coefficient, as shown in Table 6. Therefore, 
we strongly recommend using the 4 grades visual score criteria 
(consider 2+ or 3+ as positive) as the standard IHC staining scoring 
criteria to detect EGFR mutation status in NSCLC. Moreover, 
different standard could also affect the consistency. Detection by 
Direct sequencing method for E746-A750 showed more homoge- 
neous than non-Direct sequencing method (Table 5). With 
uniform standard and IHC score criteria, we can not only reduce 
the difference among different readers, but also improve the 
diagnostic value of mutation-specific antibodies for detection of 
EGFR mutations in NSCLC. 

In developing countries, especially in undeveloped areas, high- 
tech molecular-based detection techniques are difficult to access. 
However, IHC is cost-effective and widely-available, which can be 
performed on a large scale. In addition, IHC is easy to perform 
and not time-intensive, making it popular with both clinicians and 
technicians. Moreover, IHC can provide reliable results with only 
a limited amount of tissue material, such as small biopsies or 
cytological samples. However, there are some limitations to the 
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mutation-specific antibodies. Currently, the availability of only two 
antibodies would be considered insufficient for clinical application, 
as rarer, sensitizing EGFR mutations could not be detected. 
Additionally, considering a variety of IHC staining criteria were 
used in the studies, it would be necessary to establish a uniform 
immunohistochemical staining protocol. 

In conclusion, we recommend using the immunohistochemical 
method alone for detection of NSCLC EGFR mutation if results 
are positive for EGFR mutation status. If detection of mutations in 
exon 19 have a negative result after IHC, molecular-based 
analyses become necessary. However, for exon 21 mutations, we 
recommend using confirmatory molecular testing if time and 
economic resources permit. In summary, mutation-specific anti- 
bodies for immunohistochemical detection of EGFR mutation 
status is a novel cost-effective [9], and widely-available method 
that deserves further investigation. 
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